Beneath the buzz, the next-generation developer framework Ray was key in the viral model's training.
"ChatGPT combined a lot of the previous work on large language models with reinforcement as well.
Before deploying Ray, OpenAI used a hodgepodge set of custom tools built on top of "neural programmer-interpreter" model.
All these tools, Ray and JAX included, are in service to a new generation of combustion engines for the internet called large language models.
Multiple companies, both startups and giants, are building their own large language models including Meta, Hugging Face, OpenAI, and Google.